Lecture Five - STAT 212a
نویسنده
چکیده
1 Varshamov-Gilbert Lemma Lemma 1.1. Fix k ≥ 1. There exists a subset W of {0, 1} with |W | ≥ exp(k/8) such that the Hamming distance, ∆(τ, τ ′) := ∑k i=1{τi 6= τ ′ i} > k/4 for all τ, τ ′ ∈W with τ 6= τ ′. Proof. Consider a maximal subset W of {0, 1} that satisfies: ∆(τ, τ ′) ≥ k/4 for all τ, τ ′ ∈W with τ 6= τ ′. (1) The meaning of maximal here is that if one tries to expand W by adding one more element, then the constraint (1) will be violated. In other words, if we define the closed ball, B(τ, k/4) := {θ ∈ {0, 1} : ∆(θ, τ) ≤ k/4} for τ ∈ {0, 1}, then we must have ⋃ τ∈W B(τ, k/4) = {0, 1}.
منابع مشابه
Lecture Three - STAT 212a
The mutual information I(θ; X) between two random variables θ and X is defined as the Kullback-Leibler divergence between their joint distribution and the product of their marginal distributions. It is interpreted as the amount of information that X contains about θ.
متن کاملLecture Eight - STAT 212a
1 Sudakov Minoration Suppose that (T, ρ) is a finite metric space and let {X t , t ∈ T } be a stochastic process indexed by T satisfying
متن کاملLecture Six - STAT 212a
Conversely, for every -packing subset t1, . . . , tn of T , the closed balls B(ti, /2), i = 1, . . . , n are disjoing and hence every /2-cover of T must have one point in each of the balls B(ti, /2). As a result, an /2-cover of T must have at least n points. This implies that M( /2, T ) ≥ N( , T ). Lemma 1.2 (Volumetric Argument). Let T = X denote the ball in R of radius Γ centered at the origi...
متن کاملLecture Two - STAT 212a
1 f-divergences Recall from last lecture: Here is the definition of an f-divergence: Let f : (0, ∞) → R be a convex function with f (1) = 0. By virtue of convexity, both the limits f (0) := lim x↓0 f (x) and f (∞) := lim x↑∞ f (x)/x exist, although they may be equal to +∞. For two probability measures P and Q, the f-divergence D f (P ||Q) is defined by D f (P ||Q) := {q > 0} f p q dQ + f (∞)P {...
متن کامل